Augmented reality system with an active portable anchor

ABSTRACT

A method, apparatus, and augmented reality system comprising an active portable anchor used by an augmented reality device to display augmented reality information on a live view of an object. The active portable anchor is configured to output a current physical anchor position of the active portable anchor relative to an object.

CROSS-REFERENCE TO RELATED APPLICATION

This application is related to U.S. patent application Ser. No.16/032,168, entitled “Augmented Reality System with an Active PortableAnchor,” filed on Jul. 11, 2018, and issued as U.S. Pat. No. 10,839,604on Nov. 17, 2020, which is incorporated herein by reference in itsentirety.

BACKGROUND INFORMATION 1. Field

The present disclosure relates generally to an improved augmentedreality system and, in particular, to an improved method, apparatus, andsystem for an augmented reality system with active anchors.

2. Background

Augmented reality involves an interactive experience with a real-worldenvironment that is augmented by information from a computer system. Theinformation is displayed on a live view of the real-world environmentseen through an augmented reality device. The information is displayedon the live view in a manner that provides descriptions or indicatorsabout objects in the live view to a user. This information is alsoreferred to as augmented reality information. In other cases, theaugmented reality information is displayed on the live view of thereal-world environment in a manner that is seamlessly interwoven suchthat the information perceived as part of the real-world environment isseen through the augmented reality device.

Augmented reality devices are used in many applications in real-worldenvironments. For example, augmented reality devices can be used inmanufacturing or performing maintenance on an object such as anaircraft. Augmented reality devices are used by human operatorsperforming operations to assemble parts, fabricate parts, inspectassemblies, or other operations. Information used to augment a live viewof the aircraft may include, for example, a schematic diagram, graphicalindicators identifying an inconsistency, a wiring diagram, a display ofcomponents hidden under skin panels, or other types of information.

Anchors are used to describe the position of an object in the real-worldenvironment. An anchor is also referred to as a spatial anchor and marksa position with respect to the object in the real-world environment. Theanchor can be correlated to a corresponding position in a model of theobject.

Displaying augmented reality information such as graphics or hologramson a live view of an object can be more accurate using the coordinatesystem of an anchor. These anchors can be used by the augmented realitydevice to orient itself. In other words, the anchors can be used by theaugmented reality device to identify the position of the augmentedreality device for use in displaying the augmented reality informationon the live view of the object.

Currently, anchors are permanent in the real-world environment. In otherwords, an anchor is fixed for use in displaying augmented realityinformation. For example, anchor plates are a type of physical anchorand are formed or installed on an object such as a room or pipes in theroom. The positions of these anchor plates are correlated to theirrespective positions in the model of the object to provide the augmentedreality device with a reference for itself to display augmented realityinformation with a desired level of accuracy on the live view of object.

One drawback with using these anchors is the limited distance from whichan augmented reality device can be located with respect to accuracy. Asthe distance of the augmented reality device from the anchor increases,the accuracy with which an augmented reality device in displayingaugmented reality information on the live view of the object decreases.

For example, a distance of more than five meters from may result in anundesired level of accuracy for the augmented reality device to displaythe augmented reality information in positions on or proximate to theobject. As a result, the number of anchors needed for desired accuracymay be greater than possible or feasible for some objects. Further,programming anchors into a model for an augmented reality system istime-consuming.

Therefore, it would be desirable to have a method and apparatus thattake into account at least some of the issues discussed above, as wellas other possible issues. For example, it would be desirable to have amethod and apparatus that overcome a technical problem with orienting anaugmented reality device to an object in a real-world environment.

SUMMARY

In an illustrative embodiment, an apparatus comprises an active portableanchor used by an augmented reality device to display augmented realityinformation on a live view of an object. The active portable anchor isconfigured to output a current physical anchor position of the activeportable anchor relative to the object.

In another illustrative embodiment, an augmented reality systemcomprises an augmented reality device configured to receive a currentphysical anchor position of an active portable anchor relative to anobject from the active portable anchor. The augmented reality device isalso configured to determine a current model anchor position of theactive portable anchor in a model of the object using the currentphysical anchor position of the active portable anchor relative to theobject. The augmented reality device is configured to display augmentedreality information on a live view of the object in the augmentedreality device using the current model anchor position of the activeportable anchor and the model of the object. Accuracy in positioning theaugmented reality information on the live view of the object isincreased with updating the current model anchor position of the activeportable anchor in the model of the object to reflect the currentphysical anchor position of the active portable anchor relative to aphysical object position of the object.

In another illustrative embodiment, a method is present for displayingaugmented reality information on an augmented reality device. Theaugmented reality device receives a current physical anchor position ofan active portable anchor relative to an object from the active portableanchor. A current model anchor position of the active portable anchor ina model of the object is determined using the current physical anchorposition of the active portable anchor relative to the object. Theaugmented reality information is displayed in a live view of the objectin the augmented reality device using the current model anchor positionof the active portable anchor and the model of the object. Accuracy inpositioning the augmented reality information on the live view of theobject is increased with dynamically updating the current model anchorposition of the active portable anchor in the model of the object toreflect the current physical anchor position of the active portableanchor relative to a physical object position of the object.

In yet another illustrative embodiment, a method is present fororienting an augmented reality device. A current physical anchorposition of an active portable anchor is identified. The active portableanchor is used by the augmented reality device to display augmentedreality information on a live view of an object. The current physicalanchor position of the active portable anchor relative to the object isoutput.

The features and functions can be achieved independently in variousembodiments of the present disclosure or may be combined in yet otherembodiments in which further details can be seen with reference to thefollowing description and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features believed characteristic of the illustrativeembodiments are set forth in the appended claims. The illustrativeembodiments, however, as well as a preferred mode of use, furtherobjectives and features thereof, will best be understood by reference tothe following detailed description of an illustrative embodiment of thepresent disclosure when read in conjunction with the accompanyingdrawings, wherein:

FIG. 1 is a pictorial representation of a network of data processingsystems in which illustrative embodiments may be implemented;

FIG. 2 is an illustration of a block diagram of an augmented realityenvironment in accordance with an illustrative embodiment;

FIG. 3 is an illustration of a block diagram of an active portableanchor in accordance with an illustrative embodiment;

FIG. 4 is an illustration of a dataflow diagram for displaying augmentedreality information on a live view of an object in accordance with anillustrative embodiment;

FIG. 5 is an illustration of an augmented reality environment inaccordance with an illustrative embodiment;

FIG. 6 is an illustration of an active portable anchor in accordancewith an illustrative embodiment;

FIG. 7 is an illustration of a flowchart of a process for displayingaugmented reality information on a live view in accordance with anillustrative embodiment;

FIG. 8 is an illustration of a flowchart of a process for outputting acurrent physical anchor position to orient an augmented reality devicein accordance with an illustrative embodiment;

FIG. 9 is an illustration of a flowchart of a process for displayingaugmented reality information in accordance with an illustrativeembodiment;

FIG. 10 is an illustration of a flowchart of a process for orienting anaugmented reality device in accordance with an illustrative embodiment;

FIG. 11 is an illustration of a flowchart of a process for receiving acurrent physical anchor position in accordance with an illustrativeembodiment;

FIG. 12 is an illustration of a block diagram of a data processingsystem in accordance with an illustrative embodiment;

FIG. 13 is an illustration of a block diagram of an aircraftmanufacturing and service method in accordance with an illustrativeembodiment;

FIG. 14 is an illustration of a block diagram of an aircraft in which anillustrative embodiment may be implemented; and

FIG. 15 is an illustration of a block diagram of a product managementsystem in accordance with an illustrative embodiment.

DETAILED DESCRIPTION

The illustrative embodiments recognize and take into account one or moredifferent considerations. The illustrative embodiments recognize andtake into account that accuracy in positioning augmented realityinformation displayed on a live view of an aircraft is important inperforming operations on the aircraft. For example, the augmentedreality information can be displayed on the live view to performoperations such as drilling holes, reworking an inconsistency, applyinga sealant, or installing fasteners in the aircraft. In one illustrativeexample, the augmented reality information can show inconsistenciesusing visual information tags. These tags can mark the location of theinconsistencies and provide information about the inconsistencies.

The illustrative embodiments recognize and take into account that, in adynamic environment, the use of permanent anchors is currentlyinfeasible. Further, the illustrative embodiments recognize and takeinto account that techniques for programming anchors into a model foruse by an augmented reality device is time-consuming. The illustrativeembodiments recognize and take into account that the amount of set-uptime for currently used anchors is much greater than desired.

The illustrative embodiments recognize and take into account thatcurrently available techniques for calibrating or orienting an augmentedreality device do not work well in a dynamic environment in which atleast one of objects or human operators with augmented reality devicesmove. For example, the illustrative embodiments recognize and take intoaccount that using fixed point or dynamic photogrammetry is taxing inresource use in augmented reality systems and requires much effort toprogram in features such as anchor plates on the object into the modelof the object.

Further, the illustrative embodiments recognize and take into accountthat, with aircraft, the number of features that are usable for thistype of process are few enough to be infeasible. Additionally, theillustrative embodiments recognize and take into account that thecomputer-aided design data would be much greater than desired without adesired resolution. The illustrative embodiments recognize and take intoaccount that this type of technique is more suitable for smaller volumesor spaces as compared to those for an aircraft.

The illustrative embodiments also recognize and take into account thatcurrently used anchor plates are not very useful because they arerequired to be in known positions. This type of limitation makes anchorplates infeasible for use in dynamic environments in which humanoperators may move with respect to the aircraft.

The illustrative embodiments recognize and take into account that activeportable anchors can enable augmented reality devices to quickly orientthemselves to the object in an environment. The illustrative embodimentsrecognize and take into account that active portable anchors can be usedas a calibration plate similar to current anchors, allowing an augmentedreality device to recognize that an anchor is present in the real-worldenvironment. The illustrative embodiments also recognize and take intoaccount that an active portable anchor can also be configured to providean augmented reality device the current position of the active portableanchor relative to the object.

The illustrative embodiments recognize and take into account that thistype of solution can be implemented by utilizing a secondary locationsystem in the form of an external tracking system. The illustrativeembodiments recognize and take into account that the external trackingsystem operates to identify the position of the active portable anchorsand relay that information through those active portable anchors, whichin turn provide the information to augmented reality devices.

Thus, the illustrative embodiments provide a method, an apparatus, and asystem for providing an augmented reality device an ability to locateitself within an environment using active anchors that are portable. Inone illustrative example, an augmented reality system comprises anactive portable anchor, a processor system, and an augmented realitydevice. The active portable anchor is configured to output a currentphysical anchor position of the active portable anchor relative to anobject. The processor system is configured to determine the currentphysical anchor position of the active portable anchor relative to theobject using a physical anchor position of the active portable anchorand a physical object position of the object.

The augmented reality device is configured to receive the currentphysical anchor position of the active portable anchor relative to theobject from the active portable anchor. The augmented reality device isalso configured to determine a current model position of the activeportable anchor in a model of the object using the current physicalanchor position and display augmented reality information in associationon a live view of the object in the augmented reality device using thecurrent model anchor position of the active portable anchor and themodel of the object. The model anchor position of the active portableanchor in the model of the object is dynamically updated to reflect thecurrent physical anchor position of the active portable anchor relativeto the object.

The position of an object includes the location of the object describedin three dimensions. For example, the location can be described using aCartesian coordinate system. Further, the position of the object alsoincludes an orientation of the object. The orientation can be describeusing two or more vectors in the illustrative example.

With reference now to the figures and, in particular, with reference toFIG. 1, a pictorial representation of a network of data processingsystems is depicted in which illustrative embodiments may beimplemented. Network data processing system 100 is a network ofcomputers in which the illustrative embodiments may be implemented.Network data processing system 100 contains network 102, which is themedium used to provide communications links between various devices andcomputers connected together within network data processing system 100.Network 102 may include connections, such as wire, wirelesscommunication links, or fiber optic cables.

In the depicted example, server computer 104 and server computer 106connect to network 102 along with storage unit 108. In addition, clientdevices 110 connect to network 102. As depicted, client devices 110include client computer 112, client computer 114, and client computer116. Client devices 110 may be, for example, computers, workstations, ornetwork computers. In the depicted example, server computer 104 providesinformation, such as boot files, operating system images, andapplications to client devices 110. Further, client devices 110 can alsoinclude other types of client devices such as mobile phone 118, tabletcomputer 120, and smart glasses 122. In this illustrative example, someor all of client devices 110 may form an Internet of things (IOT) inwhich these physical devices can connect and exchange data.

Client devices 110 are clients to server computer 104 in this example.Network data processing system 100 may include additional servercomputers, client computers, and other devices not shown. Client devices110 connect to network 102 utilizing at least one of wired, opticalfiber, or wireless connections.

Program code located in network data processing system 100 may be storedon a computer recordable storage medium and downloaded to a dataprocessing system or other device for use. For example, program code maybe stored on a computer recordable storage medium on server computer 104and downloaded to client devices 110 over network 102 for use on clientdevices 110.

In the depicted example, network data processing system 100 is theInternet with network 102 representing a worldwide collection ofnetworks and gateways that use the Transmission ControlProtocol/Internet Protocol (TCP/IP) suite of protocols to communicatewith one another. At the heart of the Internet is a backbone ofhigh-speed data communication lines between major nodes or hostcomputers consisting of thousands of commercial, governmental,educational, and other computer systems that route data and messages. Ofcourse, network data processing system 100 also may be implemented usinga number of different types of networks. For example, network 102 may becomprised of at least one of the Internet, an intranet, a local areanetwork (LAN), a metropolitan area network (MAN), or a wide area network(WAN). FIG. 1 is intended as an example, and not as an architecturallimitation for the different illustrative embodiments.

As used herein, “a number of,” when used with reference to items, meansone or more items. For example, “a number of different types ofnetworks” is one or more different types of networks.

Further, the phrase “at least one of,” when used with a list of items,means different combinations of one or more of the listed items may beused, and only one of each item in the list may be needed. In otherwords, “at least one of” means any combination of items and number ofitems may be used from the list, but not all of the items in the listare required. The item may be a particular object, a thing, or acategory.

For example, without limitation, “at least one of item A, item B, oritem C” may include item A, item A and item B, or item C. This examplealso may include item A, item B, and item C or item B and item C. Ofcourse, any combinations of these items may be present. In someillustrative examples, “at least one of” may be, for example, withoutlimitation, two of item A; one of item B; and ten of item C; four ofitem B and seven of item C; or other suitable combinations.

In one illustrative example, human operator 124 operates an augmentedreality device, such as smart glasses 122, to perform at least one ofmanufacturing or maintenance operations on aircraft 126. As depicted,server computer 106 sends model 128 of aircraft 126 to smart glasses 122for use in displaying augmented reality information to augment a liveview of aircraft 126 for human operator 124.

In this depicted example, active portable anchor 130 and externaltracking system 132 provide information utilized to identify theposition of smart glasses 122 relative to aircraft 126. Externaltracking system 132 identifies the physical position of aircraft 126 andactive portable anchor 130. A processor system located in at least oneof active portable anchor 130, external tracking system 132, clientdevices 110, server computer 104, or server computer 106 identifies thecurrent physical anchor position of active portable anchor 130 relativeto aircraft 126. Active portable anchor 130 sends this current physicalanchor position to smart glasses 122.

As depicted, smart glasses 122 receives the current physical position ofactive portable anchor 130 from active portable anchor 130. This currentphysical anchor position of active portable anchor 130 can be receivedby smart glasses 122 in a number of different ways. For example, thecurrent physical anchor position can be received from at least one of adisplay of the current physical anchor position on active portableanchor 130 or a wireless transmission of the current physical positionfrom active portable anchor 130.

The current physical anchor position of active portable anchor 130 isused by smart glasses 122 to determine where to display augmentedreality information on a live view of aircraft 126. For example,augmented reality information may be wiring for aircraft 126 locatedbelow the surface of aircraft 126. With the current physical position,smart glasses 122 determines where to display graphics in the live viewof aircraft 126 to show the wiring in the actual location of the wiringin aircraft 126.

Smart glasses 122 determines its position relative to active portableanchor 130. Upon calibrating itself to active portable anchor 130, smartglasses 122 can process model 128 of aircraft 126 to display augmentedreality information without user input and without using photogrammetrydata of a larger model.

With respect to accuracy, active portable anchor 130 can be moved ashuman operator 124 moves to maintain a distance between smart glasses122 and active portable anchor 130 such that a desired accuracy ismaintained in displaying augmented reality information in positions onthe live view of aircraft 126. When active portable anchor 130 is moved,a new physical anchor position of active portable anchor 130 can bedynamically determined and provided to smart glasses 122.

The illustration of network data processing system 100 in the componentsfor the augmented reality system is not meant to limit the manner inwhich network data processing system 100 can implement it. In otherillustrative examples, one or more intelligent global positioninglocators may be present in addition to the ones depicted. Further, oneor more users with augmented reality devices in addition to humanoperator 124 may be present performing operations on aircraft 126.

In one illustrative example, network 102 includes communications port103. Communications port 103 is a hardware port in network 102 that isconfigured to provide for communications with other data processingsystems or devices, such as active portable anchor 130. Thecommunications can be direct or indirect with one or more other devicesfacilitating the sending or receiving of communications. In theseillustrative examples, communications port 103 is located in a networkinterface card, a switch, a router, or some other device.

As depicted, communications port 103 is configured to communicate withactive portable anchor 130 and send a current physical anchor positionof active portable anchor 130 relative to an object, such as aircraft126, to active portable anchor 130 in which the current physical anchorposition of active portable anchor 130 relative to the object isdetermined using a physical anchor position of active portable anchor130 and a physical object position of the object. Active portable anchor130 is configured to output the current physical anchor position ofactive portable anchor 130 relative to an object, such as aircraft 126.

Further, an augmented reality device, such as smart glasses 122, isconfigured to receive the current physical anchor position of activeportable anchor 130 relative to the object from active portable anchor130. Smart glasses 122 are configured to determine a current modelanchor position of active portable anchor 130 in model 128 of the objectusing the current physical anchor position and display augmented realityinformation in association on a live view of the object in the augmentedreality device using the current model anchor position of activeportable anchor 130 and model 128 of the object. The current modelanchor position of active portable anchor 130 in model 128 of the objectis dynamically updated to reflect the current physical anchor positionof active portable anchor 130 relative to the object.

Thus, a technical problem with orienting an augmented reality device toan object in a real-world environment can be overcome in network dataprocessing system 100. One or more technical solutions may provide atechnical effect of enabling an augmented reality device, such as smartglasses 122, to orient itself with an anchor, such as active portableanchor 130. In one or more technical solutions, active portable anchor130 can be moved to different locations relative to an object, such asaircraft 126, enabling maintaining a desired distance between smartglasses 122 and active portable anchor 130, which results in maintaininga desired level of accuracy in displaying augmented reality informationon a live view in smart glasses 122. As a result, one or more of thetechnical solutions using active portable anchor 130 enables humanoperator 124 to perform operations on aircraft 126 with a desired levelof accuracy when relying on the display of augmented reality informationon a live view in smart glasses 122. For example, when the operation isa manufacturing operation such as drilling a hole, the locations ofholes can be indicated in augmented reality information on the live viewof aircraft 126 with a desired level of accuracy for drilling the hole.

Turning next to FIG. 2, an illustration of a block diagram of anaugmented reality environment is depicted in accordance with anillustrative embodiment. In this illustrative example, the differentcomponents shown in block form in augmented reality environment 200 canbe implemented using components shown in network data processing system100 in FIG. 1.

Augmented reality environment 200 is an environment in which augmentedreality system 202 is configured to augment live view 204 of object 206with augmented reality information 208 seen by human operator 210utilizing augmented reality device 212. Object 206 may take a number ofdifferent forms. For example, object 206 can be aircraft 126 in FIG. 1.Further, object 206 also can be selected from a mobile platform, astationary platform, a land-based structure, an aquatic-based structure,a space-based structure, a commercial aircraft, a rotorcraft, a surfaceship, a tank, a personnel carrier, a train, a spacecraft, a spacestation, a satellite, a submarine, an automobile, a power plant, abridge, a dam, a house, a manufacturing facility, a building, afuselage, an engine, a wing, a skin panel, a landing gear assembly, amonument, and other suitable objects.

In this illustrative example, augmented reality device 212 is a physicalhardware device and is selected from one of a head-mounted display, anoptical see-through head-mounted display, a mobile phone, a tabletcomputer, smart glasses, wearable computer glasses, or some othersuitable hardware system that can be moved and held or worn by humanoperator 210.

As depicted, live view 204 may be provided by augmented reality device212 as a video feed on a display or by using transparent, see-throughdisplays or lenses, such that the user is able to see the physicalreal-world environment through the display in augmented reality device212.

For example, live view 204 can be seen on a display for augmentedreality device 212 in the form of a head-mounted display, smart glasses,or a tablet computer. Augmented reality information 208 can besuperimposed on the display in this type of augmented reality device212. In other illustrative examples, live view 204 may be providedindirectly to a display in which other information is displayed on liveview 204. Live view 204 can be provided using a camera system thatdisplays images or video on a display device in augmented reality device212.

In another illustrative example, live view 204 may be seen directly byhuman operator 210 of augmented reality device 212 with augmentedreality information 208 being displayed over what the user sees on liveview 204. When live view 204 is seen directly by the user, the detectionof live view 204 can be performed by using a sensor system, such as acamera system for augmented reality device 212.

As depicted, augmented reality information 208 augments live view 204and can take a number of different forms. For example, augmented realityinformation 208 may be selected from at least one of a schematicdiagram, a user input window, an instruction, a graphic for a hiddenstructure, or holograms for components below the surface of object 206that are invisible on live view 204 of object 206. The components maybe, for example, selected from at least one of wiring, a pipe, compositelayers, an inconsistency, a joint, a line replaceable unit, or othercomponents that may be in object 206.

In another illustrative example, augmented reality information 208 mayinclude a graphical indicator drawing attention to a portion of object206. For example, the graphical indicator may draw attention to aninconsistency such as a delamination, a crack, worn paint, an incorrectdimension, or other types of inconsistencies.

As another example, the graphical indicator may draw attention to thelocation in which an operation is to be performed. This operation maybe, for example, a drilling operation, a fastener installationoperation, an inspection operation, or some other suitable type ofoperation that may be used for manufacturing or maintenance of object206.

Further, augmented reality information 208 also may prompt humanoperator 210 to provide user input to verify that a change should bemade to resolve an inconsistency in object 206. In still anotherillustrative example, augmented reality information 208 can provide avisualization of user input made by human operator 210. For example,augmented reality information 208 may include graphical indicators thatshow locations where human operator 210 or another human operator has“marked” inconsistences on object 206.

As depicted, augmented reality system 202 includes a number of differentcomponents. In this example, augmented reality system 202 comprisesactive portable anchor 214, external tracking system 216, processorsystem 218, and augmented reality device 212.

Active portable anchor 214 is configured to output current physicalanchor position 220 of active portable anchor 214 relative to physicalobject position 224 of object 206.

As depicted, processor system 218 is a physical hardware system thatincludes one or more processor units. When more than one processor unitis present, those processor units may communicate with each other usinga communications medium. When the processor units are located in thesame data processing system, the communications medium may be a bus orbusses in the data processing system. When the processor units arelocated in different data processing systems, the communications mediummay be the bus or busses and a network. The processor system can beselected from at least one of a central processing unit (CPU), agraphics processing unit (GPU), a multicore processor, a physicsprocessing unit (PPU), a digital signal processor (DSP), a networkprocessor, or some other suitable type of processor.

In this illustrative example, processor system 218 can be located in oneor more different locations. For example, processor system 218 can belocated in at least one of active portable anchor 214, external trackingsystem 216, or in some other data processing system in augmented realityenvironment 200.

As depicted, processor system 218 is configured to determine currentphysical anchor position 220 of active portable anchor 214 relative toobject 206 using physical anchor position 222 of active portable anchor214 and physical object position 224 of object 206.

In this illustrative example, external tracking system 216 is a physicalhardware system. External tracking system 216 includes at least one of alaser tracker, a laser scanner, a camera, a displacement sensor, ameasurement sensor, a probe, or some other type of external trackingsystem.

External tracking system 216 is configured to determine physical anchorposition 222 of active portable anchor 214 and physical object position224 of object 206. Physical anchor position 222 and physical objectposition 224 are described using tracking coordinate system 226 forexternal tracking system 216.

In this depicted example, physical object position 224 can be placedinto object coordinate system 228 for object 206. A part of object 206can be designated as a reference point or origin.

For example, if object 206 is an aircraft, the origin can be the nosetip of the aircraft. Physical anchor position 222 can be converted orcorrelated to object coordinate system 228. Object coordinate system 228is the coordinate system used in model 234 of object 206.

The correlation is made such that physical anchor position 222 intracking coordinate system 226 is converted to current physical anchorposition 220 in object coordinate system 228 in which current physicalanchor position 220 of active portable anchor 214 is relative to object206. In other words, current physical anchor position 220 is relative toa reference point such as an origin for object 206 in object coordinatesystem 228.

In other words, physical anchor position 222 and physical objectposition 224 in tracking coordinate system 226 are correlated by beingconverted or transformed to use object coordinate system 228 in model234 of object 206. This conversion or transformation can be made whenthe same reference points are used for the origin of object 206. Forexample, when object 206 is an aircraft, this origin can be determined,for example, by using three or more reference points. The origin can be,for example, a nose tip, a wingtip, a window, or some other feature onor within the aircraft.

As depicted, augmented reality device 212 is configured to receivecurrent physical anchor position 220 of active portable anchor 214relative to object 206 from active portable anchor 214. Augmentedreality device 212 is configured to determine current model anchorposition 232 of active portable anchor 214 using current physical anchorposition 220 in model 234 of object 206. For example, augmented realitydevice 212 determines current model anchor position 232 of activeportable anchor 214 in model 234 of object 206 that correlates tocurrent physical anchor position 220 of active portable anchor 214relative to object 206 received from active portable anchor 214.

In this illustrative example, model 234 of object 206 can take a numberof different forms. For example, model 234 can be a partial model ofobject 206 that contains a portion of object 206 or can contain all ofobject 206. For example, when object 206 is an aircraft, model 234 canbe a wing of the aircraft or the entire aircraft. In one illustrativeexample, the partial model of object 206 is selected from at least oneof the portion of object 206 in a field of view of augmented realitydevice 212 or the portion of object 206 within a selected distance ofaugmented reality device 212.

In still another illustrative example, model 234 of object 206 may beselected from one of a plurality of partial models that are based ondividing object 206 into regions based on various criteria. The criteriamay include, for example, at least one of, a type of operation to beperformed on object 206, size limitations for the partial models,storage space available in augmented reality device 212 for a partialmodel, and other suitable types of criteria.

Augmented reality device 212 is configured to display augmented realityinformation 208 in association on live view 204 of object 206 inaugmented reality device 212 using current model anchor position 232 ofactive portable anchor 214 and model 234 of object 206. As a result,current model anchor position 232 of active portable anchor 214 in model234 of object 206 is dynamically updated to reflect current physicalanchor position 220 of active portable anchor 214 relative to object206.

In this illustrative example, computer system 236 is in communicationwith augmented reality device 212. Computer system 236 sends model 234of object 206 to augmented reality device 212 in this illustrativeexample. Computer system 236 is a physical hardware system and includesone or more data processing systems. When more than one data processingsystem is present, those data processing systems are in communicationwith each other using a communications medium. The communications mediummay be a network. The data processing systems may be selected from atleast one of a computer, a server computer, a tablet, or some othersuitable data processing system.

In one illustrative example, one or more technical solutions are presentthat overcome a technical problem with orienting an augmented realitydevice to an object in a real-world environment. As a result, one ormore technical solutions may provide a technical effect of enabling anaugmented reality device to orient itself with an anchor. One or moretechnical solutions provide an active portable anchor in which thisanchor can be moved to different locations enabling maintaining adesired distance between augmented reality device 212 and activeportable anchor 214, which results in maintaining a desired level ofaccuracy in displaying augmented reality information 208 on live view204.

As a result, augmented reality system 202 operates as a special purposecomputer system in which active portable anchor 214 in augmented realitysystem 202 enables augmented reality device 212 to display augmentedreality information 208 on live view 204 of object 206 with increasedaccuracy. For example, active portable anchor 214 is a physicalcomponent that transforms augmented reality system 202 into a specialpurpose computer system as compared to currently available generalcomputer systems that do not have active portable anchor 214.

Turning next to FIG. 3, an illustration of a block diagram of an activeportable anchor is depicted in accordance with an illustrativeembodiment. In the illustrative examples, the same reference numeral maybe used in more than one figure. This reuse of a reference numeral indifferent figures represents the same element in the different figures.

This figure depicts one manner in which active portable anchor 214 canbe implemented. In this example, active portable anchor 214 comprisesportable frame 300, communications unit 302, output system 304, andprocessor unit 306. In this example, these components enable activeportable anchor 214 to be used by an augmented reality device to displayaugmented reality information on a live view of an object. Thesecomponents operate to enable active portable anchor 214 to output acurrent physical anchor position of active portable anchor 214 relativeto an object.

As depicted, portable frame 300 is a structure that operates to support,enclose, or hold components for active portable anchor 214. At least oneof dimensions or weight of portable frame 300 is selected such thatportable frame 300 can be moved by a human operator. In some cases, thedimensions or weight can be selected such that two human operators maymove portable frame 300.

Portable frame 300 can take a number of different forms. For example,portable frame 300 can be a housing, an enclosure, a platform, or someother suitable structure. In this example, communications unit 302,output system 304, and processor unit 306 can be physically connected toportable frame 300.

As used herein, a first component “physically connected to” a secondcomponent means that the first component can be connected directly orindirectly to the second component. In other words, additionalcomponents may be present between the first component and the secondcomponent. The first component is considered to be indirectly connectedto the second component when one or more additional components arepresent between the two components. When the first component is directlyconnected to the second component, no additional components are presentbetween the two components.

As depicted, communications unit 302 can be implemented using Wi-Fi,Bluetooth, or some other wireless transmission technology using radiofrequency waves, optical signals, or other types of wirelesstransmission media. For example, “Wi-Fi” is a trademark of the Wi-Fialliance, and “Bluetooth” is a registered trademark of Bluetooth SIG,Inc.

In this illustrative example, communications unit 302 is configured toreceive physical anchor position 222 of active portable anchor 214 andphysical object position 224 of object 206 as determined by externaltracking system 216. This position information can be received using awireless communications link established by communications unit 302.

Output system 304 is physically connected to portable frame 300. Theoutput from output system 304 and active portable anchor 214 can be madeusing at least one of a wireless transmitter or a display device.

In this example, processor unit 306 is physically connected to portableframe 300 and in communication with communications unit 302 and outputsystem 304. Processor unit 306 is configured to determine the currentphysical anchor position of the active portable anchor relative to aphysical object position of an object using the physical anchor positionand the physical object position; and output the current physical anchorposition of the active portable anchor relative to the physical objectposition of the object using the output system.

The illustration of augmented reality environment 200 and the differentcomponents in FIG. 2 and FIG. 3 are not meant to imply physical orarchitectural limitations to the manner in which an illustrativeembodiment may be implemented. Other components in addition to or inplace of the ones illustrated may be used. Some components may beunnecessary. Also, the blocks are presented to illustrate somefunctional components. One or more of these blocks may be combined,divided, or combined and divided into different blocks when implementedin an illustrative embodiment.

Turning next to FIG. 4, an illustration of a dataflow diagram fordisplaying augmented reality information on a live view of an object isdepicted in accordance with an illustrative embodiment. As depicted,data flow for displaying augmented reality information on a live view ofobject 400 is depicted in accordance with an illustrative embodiment. Inthis illustrative example, smart glasses 402 are example of animplementation for augmented reality device 212 in FIG. 2. Smart glasses402 receives model 404 of object 400 from computer 405. Model 404 isused by smart glasses 402 to display augmented reality information onthe live view of object 400. As depicted, graphics, information, andother data from model 404 can be used to create augmented realityinformation or be displayed as augmented reality information.

In positioning augmented reality information 418 on the live view ofobject 400, smart glasses 402 orients itself with respect to activeportable anchor 406. In this example, the orientation is performed byrecognizing active portable anchor 406 as a reference for use inidentifying the position of smart glasses 402 and correlating model 404of object 400 with the live view of object 400. This recognition can bemade when active portable anchor 406 is in the field of view of smartglasses 402.

Smart glasses 402 use photogrammetry sensors and processes to identifyactive portable anchor 406. This identification can be based on at leastone of shape, dimensions, or other visual characteristics of activeportable anchor 406. In this illustrative example, active portableanchor 406 also provides its position to smart glasses 402. The positionof active portable anchor 406 is received from active portable anchor406 in current position data 414 and used to display augmented realityinformation 418 on the live view of active portable anchor 406.

In this illustrative example, active portable anchor 406 is moveableduring the operation of smart glasses 402. The current position ofactive portable anchor 406 can be determined dynamically using externaltracking system 408.

In this illustrative example, external tracking system 408 includespositioner 410 and object positioner probe 411. Object positioner probe411 may be manipulated by at least one of a human user or a robot totouch different points on object 400. Positioner 410 recordsmeasurements based on object positioner probe 411 touching object 400.Object positioner probe 411 also can be used make measurements of activeportable anchor 406. These measurements may be used to identify aphysical anchor position for active portable anchor 406 and a physicalobject position for object 400 that form position information 412.

Position information 412 is sent to active portable anchor 406. Activeportable anchor 406 generates current position data 414 using positioninformation 412. Current position data 414 includes a current physicalanchor position of active portable anchor 406 relative to object 400.Active portable anchor 406 sends current position data 414 to smartglasses 402.

As depicted, smart glasses 402 identify the current physical anchorposition of active portable anchor 406 relative to object 400 in currentposition data 414. The position of active portable anchor 406 in model404 is updated such that the current model anchor position for activeportable anchor 406 corresponds to the current physical anchor positionfor active portable anchor 406 relative to object 400.

Smart glasses 402 orients itself with respect to object 400 using activeportable anchor 406. With this orientation, smart glasses 402 identifythe position of smart glasses 402 relative to active portable anchor406.

Smart glasses 402 display augmented reality information 418 on the liveview of object 400. The positioning of augmented reality information 418on the live view can be performed with greater accuracy when activeportable anchor 406 is within a desired distance of smart glasses 402.If smart glasses 402 move, active portable anchor 406 can also be moved.

If active portable anchor 406 is moved, updates to the position in model404 can be performed dynamically using external tracking system 408 toidentify the position of active portable anchor 406 and send updatedposition information to active portable anchor 406.

With reference to FIG. 5, an illustration of an augmented realityenvironment is depicted in accordance with an illustrative embodiment.Augmented reality environment 501 is an example of one implementationfor augmented reality environment 200 shown in block form in FIG. 2.

In this illustrative example, aircraft 500 is an example of object 206in FIG. 2. Further, active portable anchor 502 is positioned relative toaircraft 500. As depicted, external tracking system 504 identifies theposition of aircraft 500 and active portable anchor 502. Externaltracking system 504 comprises intelligent global positioning system(iGPS) locator 506, intelligent global positioning system (iGPS) locator508, and intelligent global positioning system (iGPS) locator 510.

These intelligent global positioning system locators include lasertransmitters that scan a volume of space encompassing aircraft 500 andactive portable anchor 502 from different locations to identifypositions of these two objects. The positions include locations andorientations based on the coordinate system for external tracking system504.

In this depicted example, nose tip 512 of aircraft 500 is selected as areference point for aircraft 500. As depicted, nose tip 512 has aposition that includes location L1(100,200,50) and an orientationdescribed as X-Vector X1(0.7,0.7,0.14) and Z-Vector Z1(−0.19, 0.28,0.98). Top 514 of active portable anchor 502 is selected as a referencepoint for active portable anchor 502 and has a position that includeslocation L2(60,80,10) and an orientation described as X-VectorX2(0.44,−0.88,0.15) and Z-Vector Z2(0.1, −0.1, 0.99).

The location of active portable anchor 502 relative to aircraft 500 canbe identified by subtracting the position of nose tip 512 from theposition of top 514. The relative location is identified asL2−L1=relative location RL(40,120,40). The relative orientation RL(X′Z′)is X′=X2−X2=(0.26,1.58,−0.01) and Z′=Z2−Z1=(−0.29,0.38,−0.01)

Subtracting these positions provides a relative location in threedimensions and a relative orientation for active portable anchor 502that is relative to aircraft 500. This position of active portableanchor 502 can be used to update the model anchor position for activeportable anchor 502 in a model of aircraft 500.

In this depicted example, the nose tip in the model corresponding tonose tip 512 is at the origin for the model of the aircraft having alocation LM(0,0,0) with an orientation of XM=X−Vector (1,0,0) andZM=Z−Vector (0,0,1). In this example, the current model anchor positionin the model can be updated to reflect the position of the activeportable anchor in the real-world environment. For example, the anchorlocation (AL) in the model is LM(0,0,0)+RL(40,120,40)=AL(40,120,40) andthe anchor orientation (AO) is AO(X′,Z′), whereinXA=(1,0,0)+(0.26,1.58,−0.01)=(1.26,1.58,−0.01) andZA=(0,0,1)+(−0.29,0.38,−0.01)=(−0.29,0.38,0.99).

With reference next to FIG. 6, an illustration of an active portableanchor is depicted in accordance with an illustrative embodiment. Inthis illustrative example, active portable anchor 600 is an example ofone implementation for active portable anchor 214 shown in block form inFIG. 2 and FIG. 3.

As depicted, active portable anchor 600 has portable frame 606. Flatplate 608 is physically connected to portable frame 606. Flat plate 608has a shape that an augmented reality device will recognize as an anchorfor use in orienting itself.

Display device 604 is physically connected to flat plate 608 andprovides an ability to display the position of active portable anchor600. Display device 604 can display, for example, a barcode, a matrixbarcode, or some other suitable type of format that visually conveys theposition of active portable anchor 600. Antennas 610 are configured toreceive position information from an external tracking system.

The different processes in the flowcharts in FIGS. 7-9 can beimplemented in hardware, software, or both. When implemented insoftware, the processes can take the form of program code that is run byone of more processor units located in one or more hardware devices inone or more computer systems.

With reference to FIG. 7, an illustration of a flowchart of a processfor displaying augmented reality information on a live view is depictedin accordance with an illustrative embodiment. In this depicted example,the process in FIG. 7 can be implemented in one or more components inaugmented reality system 202 in FIG. 2. For example, the differentoperations can be implemented by processor system 218, which containsprocessors that can be located in the different hardware components inaugmented reality system 202 and augmented reality device 212 in FIG. 2.

The process begins by an external tracking system determining a physicalanchor position of an active portable anchor and a physical objectposition of an object (operation 700). In this illustrative example,operation 700 can be performed by one or more processor units inexternal tracking system 216 in FIG. 2. These two positions, thephysical anchor position and the physical object position, are in atracking coordinate system for the external tracking system.

A processor system generates current position data (operation 702). Thecurrent position data contains the current physical anchor position ofthe active portable anchor relative to the object. This current physicalanchor position can be identified by subtracting the physical objectposition from the physical anchor position. Operation 702 can beperformed by one or more processor units in the processor system locatedin at least one of the external tracking system and the active portableanchor.

The active portable anchor outputs the current position data (operation704). In the illustrative example, the output can be in a form selectedfrom at least one of a wireless transmission or a visual display of thecurrent position data.

The processor system updates a model of the object with the currentposition data operation 706). In operation 704, the process updates themodel to reflect a current model anchor position for the active portableanchor relative to the object in the model.

In this example, one or more processor units in the processor systemperform this operation and can be located in at least one of theaugmented reality device or a computer that stores or sends the model ofthe object to the augmented reality device, or on some other dataprocessing and the augmented reality system.

In operation 706, the position of the active physical anchor in themodel of the object is updated to reflect the current position of theactive portable anchor in the real-world environment.

An augmented reality device orients itself with the active portableanchor in a real-world environment (operation 708). The processterminates thereafter. In this illustrative example, the augmentedreality device receives and uses the current physical anchor positionoutput by the active portable anchor to correspond the model of theobject on a live view of the object in the real-world environment. Theaugmented reality device displays augmented reality information withincreased accuracy as compared to currently used.

With reference next to FIG. 8, an illustration of a flowchart of aprocess for outputting a current physical anchor position to orient anaugmented reality device is depicted in accordance with an illustrativeembodiment. The process depicted in this flowchart can be implemented inactive portable anchor 214 in FIG. 2.

The process begins by identifying a current physical anchor position ofan active portable anchor (operation 800). The active portable anchor isused by an augmented reality device to display augmented realityinformation on a live view of an object.

In one illustrative example, the identification performed in operation800 comprises determining, by the active portable anchor, the currentphysical anchor position of the active portable anchor relative to theobject using a physical anchor position of the active portable anchorand a physical object position of the object. In another example, theidentifying step comprises determining, by the active portable anchor,the current model anchor position of the active portable anchor in themodel of the object that correlates to the current physical anchorposition of the active portable anchor relative to the object receivedfrom the active portable anchor. In still another illustrative example,the identifying step comprises determining, by an external tracker, thecurrent physical anchor position of the active portable anchor relativeto the object using a physical anchor position of the active portableanchor and a physical object position of the object.

The process outputs the current physical anchor position of the activeportable anchor relative to an object (operation 802). The processterminates thereafter. This current physical anchor position can bedynamically updated when the active portable anchor is moved to maintaina desired distance from an augmented reality device.

Turning next to FIG. 9, an illustration of a flowchart of a process fordisplaying augmented reality information is depicted in accordance withan illustrative embodiment. The process in FIG. 9 can be implemented inaugmented reality device 212 in FIG. 2.

The process begins by receiving a current physical anchor position of anactive portable anchor relative to an object from the active portableanchor (operation 900). The process determines a current model positionof the active portable anchor in a model of the object using the currentphysical anchor position of the active portable anchor relative to theobject (operation 902). The process updates the model with the currentmodel position of the active portable anchor (operation 904).

The process displays augmented reality information on a live view of theobject in an augmented reality device using the current model positionof the active portable anchor and the model of the object (operation906). The process terminates thereafter.

Accuracy in positioning augmented reality information on a live view ofthe object is increased with dynamically updating the current positionof the active portable anchor in the model of the object to reflect thecurrent physical anchor position of the active portable anchor relativeto the physical object position of the object.

With reference next to FIG. 10, an illustration of a flowchart of aprocess for orienting an augmented reality device is depicted inaccordance with an illustrative embodiment. The process in FIG. 10 canbe implemented in one or more components in augmented reality system202. For example, the different operations can be implemented by one ormore processor units in augmented reality device 212 in FIG. 2.

The process begins by obtaining a current physical anchor position of anactive portable anchor from the active portable anchor (operation 1000).In operation 1000, the current physical anchor position can be displayedon a display device on the active portable anchor. The current physicalanchor position can be obtained by the augmented reality device from thedisplay device. In another illustrative example, the current physicalanchor position can be obtained by the augmented reality device from awireless transmission received by the augmented reality device from theactive portable anchor.

The process sets a local coordinate system for the augmented realitydevice based on the current physical anchor position of the activeportable anchor (operation 1002). This position includes a location inthree-dimensional space and an orientation.

The process identifies a current model anchor position of the activeportable anchor in a model of an object (operation 1004). This currentmodel anchor position is in the coordinate system used by the model. Theprocess loads and repositions geometries in the model for display tomatch the local coordinate system (operation 1006). The processterminates thereafter. In operation 1006, the augmented reality deviceloads augmented reality information from the model for display on a liveview of the object with the desired position on the live view.

With reference next to FIG. 11, an illustration of a flowchart of aprocess for receiving a current physical anchor position is depicted inaccordance with an illustrative embodiment. The process in FIG. 11 canbe implemented in augmented reality device 212 in FIG. 2.

The process begins by identifying more than one active portable anchorsin a group of active portable anchors for an object (operation 1100). Inthis example, the group of active portable anchors is a plurality ofactive portable anchors for the object. The process selects a number ofactive portable anchors from the group of active portable anchors(operation 1102). As used herein, a “number of,” when used withreference to items, means one or more items. For example, a number ofactive portable anchors is one or more active portable anchors.

The selection in operation 1102 can be made a number of different ways.For example, the selection of the closest active portable anchor can bebased on the closest one in a field of view of an augmented realitydevice. In another example, more than one active portable anchors areused in a calibration and location is cross referenced using more thanone anchor origin. A bias or heavier weighting can be given to thecloser anchor.

The process receives a selected number of current physical anchorpositions from a selected number of current active portable anchors inthe plurality of active portable anchors (operation 1104). The processterminates thereafter. For example, the augmented reality device canreceive a single current anchor position of the closest active portableanchor. In yet another illustrative example, the augmented realitydevice receives a selected plurality of current physical anchorpositions from a plurality of current active portable anchors.

The flowcharts and block diagrams in the different depicted embodimentsillustrate the architecture, functionality, and operation of somepossible implementations of apparatuses and methods in an illustrativeembodiment. In this regard, each block in the flowcharts or blockdiagrams can represent at least one of a module, a segment, a function,or a portion of an operation or step. For example, one or more of theblocks can be implemented as program code, hardware, or a combination ofthe program code and hardware. When implemented in hardware, thehardware may, for example, take the form of integrated circuits that aremanufactured or configured to perform one or more operations in theflowcharts or block diagrams. When implemented as a combination ofprogram code and hardware, the implementation may take the form offirmware. Each block in the flowcharts or the block diagrams may beimplemented using special purpose hardware systems that perform thedifferent operations or combinations of special purpose hardware andprogram code run by the special purpose hardware.

In some alternative implementations of an illustrative embodiment, thefunction or functions noted in the blocks may occur out of the ordernoted in the figures. For example, in some cases, two blocks shown insuccession may be performed substantially concurrently, or the blocksmay sometimes be performed in the reverse order, depending upon thefunctionality involved. Also, other blocks may be added in addition tothe illustrated blocks in a flowchart or block diagram.

Turning now to FIG. 12, an illustration of a block diagram of a dataprocessing system is depicted in accordance with an illustrativeembodiment. Data processing system 1200 may be used to implement servercomputer 104, server computer 106, client devices 110, augmented realitydevice 212, active portable anchor 214, external tracking system 216,computer system 236, smart glasses 402, computer 405, active portableanchor 406, and any other data processing system that may be used toperform operations on the different illustrative examples. As depicted,data processing system 1200 includes communications framework 1202,which provides communications between processor unit 1204, memory 1206,persistent storage 1208, communications unit 1210, input/output (I/O)unit 1212, and display 1214. In this example, communications framework1202 may take the form of a bus system.

Processor unit 1204 serves to execute instructions for software that maybe loaded into memory 1206. Processor unit 1204 may be a number ofprocessors, a multi-processor core, or some other type of processor,depending on the particular implementation.

Memory 1206 and persistent storage 1208 are examples of storage devices1216. A storage device is any piece of hardware that is capable ofstoring information, such as, for example, without limitation, at leastone of data, program code in functional form, or other suitableinformation either on a temporary basis, a permanent basis, or both on atemporary basis and a permanent basis. Storage devices 1216 may also bereferred to as computer-readable storage devices in these illustrativeexamples. Memory 1206, in these examples, may be, for example, arandom-access memory or any other suitable volatile or non-volatilestorage device. Persistent storage 1208 may take various forms,depending on the particular implementation.

For example, persistent storage 1208 may contain one or more componentsor devices. For example, persistent storage 1208 may be a hard drive, asolid-state drive (SSD), a flash memory, a rewritable optical disk, arewritable magnetic tape, or some combination of the above. The mediaused by persistent storage 1208 also may be removable. For example, aremovable hard drive may be used for persistent storage 1208.

Communications unit 1210, in these illustrative examples, provides forcommunications with other data processing systems or devices. In theseillustrative examples, communications unit 1210 is a network interfacecard.

Input/output unit 1212 allows for input and output of data with otherdevices that may be connected to data processing system 1200. Forexample, input/output unit 1212 may provide a connection for user inputthrough at least one of a keyboard, a mouse, or some other suitableinput device. Further, input/output unit 1212 may send output to aprinter. Display 1214 provides a mechanism to display information to auser.

Instructions for at least one of the operating system, applications, orprograms may be located in storage devices 1216, which are incommunication with processor unit 1204 through communications framework1202. The processes of the different embodiments may be performed byprocessor unit 1204 using computer-implemented instructions, which maybe located in a memory, such as memory 1206.

These instructions are referred to as program code, computer usableprogram code, or computer-readable program code that may be read andexecuted by a processor in processor unit 1204. The program code in thedifferent embodiments may be embodied on different physical orcomputer-readable storage media, such as memory 1206 or persistentstorage 1208.

Program code 1218 is located in a functional form on computer-readablemedia 1220 that is selectively removable and may be loaded onto ortransferred to data processing system 1200 for execution by processorunit 1204. Program code 1218 and computer-readable media 1220 formcomputer program product 1222 in these illustrative examples. In theillustrative example, computer-readable media 1220 is computer-readablestorage media 1224.

In these illustrative examples, computer-readable storage media 1224 isa physical or tangible storage device used to store program code 1218rather than a medium that propagates or transmits program code 1218.

Alternatively, program code 1218 may be transferred to data processingsystem 1200 using a computer-readable signal media. Thecomputer-readable signal media may be, for example, a propagated datasignal containing program code 1218. For example, the computer-readablesignal media may be at least one of an electromagnetic signal, anoptical signal, or any other suitable type of signal. These signals maybe transmitted over at least one of communications links, such aswireless communications links, optical fiber cable, coaxial cable, awire, or any other suitable type of communications link.

The different components illustrated for data processing system 1200 arenot meant to provide architectural limitations to the manner in whichdifferent embodiments may be implemented. The different illustrativeembodiments may be implemented in a data processing system includingcomponents in addition to or in place of those illustrated for dataprocessing system 1200. Other components shown in FIG. 12 can be variedfrom the illustrative examples shown. The different embodiments may beimplemented using any hardware device or system capable of runningprogram code 1218.

Illustrative embodiments of the disclosure may be described in thecontext of aircraft manufacturing and service method 1300 as shown inFIG. 13 and aircraft 1400 as shown in FIG. 14. Turning first to FIG. 13,an illustration of a block diagram of an aircraft manufacturing andservice method is depicted in accordance with an illustrativeembodiment. During pre-production, aircraft manufacturing and servicemethod 1300 may include specification and design 1302 of aircraft 1400in FIG. 14 and material procurement 1304.

During production, component and subassembly manufacturing 1306 andsystem integration 1308 of aircraft 1400 in FIG. 14 takes place.Thereafter, aircraft 1400 in FIG. 14 may go through certification anddelivery 1310 in order to be placed in service 1312. While in service1312 by a customer, aircraft 1400 in FIG. 14 is scheduled for routinemaintenance and service 1314, which may include modification,reconfiguration, refurbishment, and other maintenance or service. Forexample, aircraft inspections during routine maintenance and service1314 can include operations to maintain or increase at least one ofsafety of flight or availability of aircraft 1400 in FIG. 14.

Each of the processes of aircraft manufacturing and service method 1300may be performed or carried out by a system integrator, a third party,an operator, or some combination thereof. In these examples, theoperator may be a customer. For the purposes of this description, asystem integrator may include, without limitation, any number ofaircraft manufacturers and major-system subcontractors; a third partymay include, without limitation, any number of vendors, subcontractors,and suppliers; and an operator may be an airline, a leasing company, amilitary entity, a service organization, and so on.

With reference now to FIG. 14, an illustration of an aircraft isdepicted in which an illustrative embodiment may be implemented. In thisexample, aircraft 1400 is produced by aircraft manufacturing and servicemethod 1300 in FIG. 13 and may include airframe 1402 with plurality ofsystems 1404 and interior 1406. Examples of systems 1404 include one ormore of propulsion system 1408, electrical system 1410, hydraulic system1412, and environmental system 1414. Any number of other systems may beincluded. Although an aerospace example is shown, different illustrativeembodiments may be applied to other industries, such as the automotiveindustry.

Apparatuses and methods embodied herein may be employed during at leastone of the stages of aircraft manufacturing and service method 1300 inFIG. 13.

In one illustrative example, components or subassemblies produced incomponent and subassembly manufacturing 1306 in FIG. 13 may befabricated or manufactured in a manner similar to components orsubassemblies produced while aircraft 1400 is in service 1312 in FIG.13. As yet another example, one or more apparatus embodiments, methodembodiments, or a combination thereof may be utilized during productionstages, such as component and subassembly manufacturing 1306 and systemintegration 1308 in FIG. 13. One or more apparatus embodiments, methodembodiments, or a combination thereof may be utilized while aircraft1400 is in service 1312, during maintenance and service 1314 in FIG. 13,or both. For example, augmented reality system 202 in FIG. 2 can beutilized during at least one of component and subassembly manufacturing1306, system integration 1308, certification and delivery 1310, inservice 1312, or maintenance and service 1314. With the use of activeportable anchor 214, the display of augmented reality information on alive view to human operator may be performed in a manner that allows thehuman operator to move more freely and maintain a desired level ofaccuracy in displaying augmented reality information as compared tocurrently available techniques. A number of the different illustrativeembodiments may substantially expedite the assembly of aircraft 1400,reduce the cost of aircraft 1400, or both expedite the assembly ofaircraft 1400 and reduce the cost of aircraft 1400.

Turning now to FIG. 15, an illustration of a block diagram of a productmanagement system is depicted in accordance with an illustrativeembodiment. Product management system 1500 is a physical hardwaresystem. In this illustrative example, product management system 1500 mayinclude at least one of manufacturing system 1502 or maintenance system1504.

Manufacturing system 1502 is configured to manufacture products, such asaircraft 1400 in FIG. 14. As depicted, manufacturing system 1502includes manufacturing equipment 1506. Manufacturing equipment 1506includes at least one of fabrication equipment 1508 or assemblyequipment 1510.

Fabrication equipment 1508 is equipment that may be used to fabricatecomponents for parts used to form aircraft 1400 in FIG. 14. For example,fabrication equipment 1508 may include machines and tools. Thesemachines and tools may be at least one of a drill, a hydraulic press, afurnace, a mold, a composite tape laying machine, a vacuum system, alathe, or other suitable types of equipment. Fabrication equipment 1508may be used to fabricate at least one of metal parts, composite parts,semiconductors, circuits, fasteners, ribs, skin panels, spars, antennas,or other suitable types of parts.

Assembly equipment 1510 is equipment used to assemble parts to formaircraft 1400 in FIG. 14. In particular, assembly equipment 1510 may beused to assemble components and parts to form aircraft 1400. Assemblyequipment 1510 also may include machines and tools. These machines andtools may be at least one of a robotic arm, a crawler, a fasterinstallation system, a rail-based drilling system, or a robot. Assemblyequipment 1510 may be used to assemble parts such as seats, horizontalstabilizers, wings, engines, engine housings, landing gear systems, andother parts for aircraft 1400.

In this illustrative example, maintenance system 1504 includesmaintenance equipment 1512. Maintenance equipment 1512 may include anyequipment needed to perform maintenance on aircraft 1400 in FIG. 14.Maintenance equipment 1512 may include tools for performing differentoperations on parts on aircraft 1400. These operations may include atleast one of disassembling parts, refurbishing parts, inspecting parts,reworking parts, manufacturing replacement parts, or other operationsfor performing maintenance on aircraft 1400. These operations may be forroutine maintenance, inspections, upgrades, refurbishment, or othertypes of maintenance operations.

In the illustrative example, maintenance equipment 1512 may includeultrasonic inspection devices, x-ray imaging systems, vision systems,drills, crawlers, and other suitable devices. In some cases, maintenanceequipment 1512 may include fabrication equipment 1508, assemblyequipment 1510, or both to produce and assemble parts that may be neededfor maintenance.

Product management system 1500 also includes control system 1514.Control system 1514 is a hardware system and may also include softwareor other types of components. Control system 1514 is configured tocontrol the operation of at least one of manufacturing system 1502 ormaintenance system 1504. In particular, control system 1514 may controlthe operation of at least one of fabrication equipment 1508, assemblyequipment 1510, or maintenance equipment 1512.

The hardware in control system 1514 may include computers, circuits,networks, and other types of equipment. The control may take the form ofdirect control of manufacturing equipment 1506. For example, robots,computer-controlled machines, and other equipment may be controlled bycontrol system 1514. In other illustrative examples, control system 1514may manage operations performed by human operators 1516 in manufacturingor performing maintenance on aircraft 1400. For example, control system1514 may assign tasks, provide instructions, display models, or performother operations to manage operations performed by human operators 1516.In these illustrative examples, augmented reality system 202 in FIG. 2may be implemented in control system 1514 to manage at least one of themanufacturing or maintenance of aircraft 1400 in FIG. 14.

For example, augmented reality system 202 in FIG. 2 operates to displayaugmented reality information to human operators performing at least oneof manufacturing or maintenance of a product such as aircraft 1400 inFIG. 14. The augmented reality information may present instructions toperform at least one of manufacturing or maintenance operations. Asanother example, the augmented reality information may identify workorders or tasks to be performed at different locations on aircraft 1400.Further, the augmented reality information displayed may guide humanoperators to positions on aircraft 1400 where operations should beperformed.

Further, augmented reality system 202 in FIG. 2 can also be used toreceive input from human operators. For example, the human operators mayrecord inconsistencies identified from visual inspections usingaugmented reality system 202.

In the different illustrative examples, human operators 1516 may operateor interact with at least one of manufacturing equipment 1506,maintenance equipment 1512, or control system 1514. This interaction maybe performed to manufacture aircraft 1400 in FIG. 14.

Of course, product management system 1500 may be configured to manageother products other than aircraft 1400 in FIG. 14. Although productmanagement system 1500 has been described with respect to manufacturingin the aerospace industry, product management system 1500 may beconfigured to manage products for other industries. For example, productmanagement system 1500 can be configured to manufacture products for theautomotive industry as well as any other suitable industries.

Thus, one or more illustrative examples utilize active portable anchorsthat are placed on or proximate an object. These active portable anchorsenable augmented reality devices to quickly orient themselves to theobject in a real-world environment. Further, the active portable anchorscan be used as a calibration plate similar to current anchors. In otherwords, an augmented reality device can recognize that an anchor ispresent in the real-world environment and can use this anchor inorienting itself to display augmented reality information. The activeportable anchor also provides an augmented reality device the currentanchor position of the active portable anchor relative to the object.

In this manner, in one or more technical solutions, the augmentedreality device can locate itself more efficiently through using theactive portable anchor and more accurately display augmented realityinformation in positions with respect to the object. For example,graphical indicators for locations in which holes are to be formed inthe object can be displayed as augmented reality information on a liveview of the object in a manner that is more accurate than with currentlyavailable systems. The active portable anchor can be moved to anotherposition with the active portable anchor being updated to send this newposition to the augmented reality device.

As a result, the augmented reality system in the illustrative examplesoperates as a special purpose computer system in which an activeportable anchor in the augmented reality system enables an augmentedreality device to display augmented reality information on a live viewof an object with increased accuracy. In particular, the active portableanchor is a physical component that transforms the augmented realitysystem into a special purpose computer system as compared to currentlyavailable general computer systems that do not have the active portableanchor.

The illustrative embodiments recognize and take into account that thistype of solution can be implemented by utilizing a secondary locationsystem in the form of an external tracking system. The illustrativeembodiments recognize and take into account that the external trackingsystem operates to identify the location of the active portable anchorsand relay that information through those active portable anchors, whichin turn provide the information to augmented reality devices. Further,accuracy in positioning the augmented reality information on the liveview of the object is increased with updating the current model anchorposition of the active portable anchor in the model of the object toreflect a change in the current physical anchor position of the activeportable anchor relative to the object from a movement of the activeportable anchor.

The description of the different illustrative embodiments has beenpresented for purposes of illustration and description and is notintended to be exhaustive or limited to the embodiments in the formdisclosed. The different illustrative examples describe components thatperform actions or operations. In an illustrative embodiment, acomponent may be configured to perform the action or operationdescribed. For example, the component may have a configuration or designfor a structure that provides the component an ability to perform theaction or operation that is described in the illustrative examples asbeing performed by the component.

Many modifications and variations will be apparent to those of ordinaryskill in the art. Further, different illustrative embodiments mayprovide different features as compared to other desirable embodiments.The embodiment or embodiments selected are chosen and described in orderto best explain the principles of the embodiments, the practicalapplication, and to enable others of ordinary skill in the art tounderstand the disclosure for various embodiments with variousmodifications as are suited to the particular use contemplated.

What is claimed is:
 1. An apparatus that comprises: an augmented realitydevice that comprises an active portable anchor that comprises anantenna and a display device connected to a plate connected to a frameconfigured to output a current physical anchor position of the activeportable anchor relative to an object to a model of the object in acomputer system; augmented reality information from the model for aportion of the object displayed on a live view of the portion of theobject, such that the active portable anchor is configured to output thecurrent physical anchor position of the active portable anchor relativeto the object based upon a physical object position of the object, suchthat the portion of the object comprises a tangible feature in a fieldof view of the augmented reality device.
 2. The apparatus of claim 1,wherein the active portable anchor is configured to: receive a physicalanchor position of the active portable anchor and the physical objectposition of the object as determined by an external tracking system thatcomprises a number of positioning locators: located off of and away fromthe object and the active portable anchor, and such that the externaltracking system is configured to scan a volume that encompasses theobject and the active portable anchor; and determine the currentphysical anchor position of the active portable anchor relative to theobject based on the physical anchor position of the active portableanchor and the physical object position of the object.
 3. The apparatusof claim 2, wherein the active portable anchor comprises: a portableframe; a communications unit physically connected to the portable frame,wherein the communications unit is configured to receive the physicalanchor position of the active portable anchor and the physical objectposition of the object as determined by the external tracking system; anoutput system physically connected to the portable frame; and aprocessor unit physically connected to the portable frame and incommunication with the communications unit and the output system,wherein the processor unit is configured to determine the currentphysical anchor position of the active portable anchor relative to thephysical object position of the object using the physical anchorposition and the physical object position; and output the currentphysical anchor position of the active portable anchor relative to thephysical object position of the object using the output system.
 4. Theapparatus of claim 2, wherein the external tracking system comprises aglobal positioning system.
 5. The apparatus of claim 3, wherein theoutput system comprises a wireless transmitter and the active portableanchor is within 5 meters of the augmented reality device.
 6. Theapparatus of claim 4, wherein the external tracking system is selectedfrom at least one of a laser tracker, a laser scanner, a camera, adisplacement sensor, a measurement sensor, or a probe.
 7. The apparatusof claim 1, further comprising the augmented reality device configuredto: receive the current physical anchor position of the active portableanchor relative to the object output by the active portable anchor; anddisplay the augmented reality information on the live view of the objectusing the current physical anchor position to place the augmentedreality information on the live view of the object.
 8. The apparatus ofclaim 7, wherein the active portable anchor has a shape that isrecognized by the augmented reality device as an anchor for displayingthe augmented reality information on the live view.
 9. The apparatus ofclaim 7 further comprising: a group of active portable anchors for theobject, wherein the active portable anchor and the group of activeportable anchors are a plurality of active portable anchors for theobject, and wherein the augmented reality device receives a selectednumber of current physical anchor positions from a selected number ofcurrent active portable anchors in the plurality of active portableanchors.
 10. The apparatus of claim 1, wherein the object is selectedfrom a group comprising a mobile platform, a stationary platform, aland-based structure, an aquatic-based structure, a space-basedstructure, an aircraft, a commercial aircraft, a rotorcraft, a surfaceship, a tank, a personnel carrier, a train, a spacecraft, a spacestation, a satellite, a submarine, an automobile, a power plant, abridge, a dam, a house, a manufacturing facility, a building, afuselage, an engine, a wing, a skin panel, a landing gear assembly, anda monument.
 11. An augmented reality system that comprises: an augmentedreality device configured to receive a current physical anchor positionof an active portable anchor relative to an object from the activeportable anchor, such that the object comprises a tangible feature;determine, using the current physical anchor position of the activeportable anchor relative to the object, a current model anchor positionof the active portable anchor in a model of the object in a computersystem; and display augmented reality information from the model for aportion of the object displayed on a live view of the object in theaugmented reality device based upon the current model anchor position ofthe active portable anchor and the model of the object, such that anaccuracy in a position of the augmented reality information from themodel for the portion of the object displayed on the live view of theportion of the object increases based upon an update of the currentmodel anchor position of the active portable anchor in the model of theobject that reflects a change in the current physical anchor position ofthe active portable anchor relative to the object based upon a movementof the active portable anchor.
 12. The augmented reality system of claim11, wherein the model of the object is a partial model of the objectthat comprises the portion of the object, and a distance from the activeportable anchor to the augmented reality device remains five meters orless.
 13. The augmented reality system of claim 12, wherein the partialmodel of the object is selected from the portion of the object in afield of view of the augmented reality device and the portion of theobject within a selected distance of the augmented reality device.
 14. Amethod for displaying augmented reality information about a portion ofan object on a live view of the portion of the object on an augmentedreality device, the method comprising: receiving, by the augmentedreality device, a current physical anchor position of an active portableanchor relative to the portion of the object in the live view from theactive portable anchor in a field of view of the augmented realitydevice, such that the object comprises a tangible feature; determining,by the augmented reality device, a current model anchor position of theactive portable anchor in a model of the object in a computer system viascanning a volume that encompasses the object and the active portableanchor and using the current physical anchor position of the activeportable anchor relative to the object; and displaying, by the augmentedreality device, the augmented reality information on the live view ofthe object in the augmented reality device using the current modelanchor position of the active portable anchor and the model of theobject, wherein accuracy in positioning the augmented realityinformation on the live view of the object is increased with dynamicallyupdating the current model anchor position of the active portable anchorin the model of the object to reflect the current physical anchorposition of the active portable anchor relative to a physical objectposition of the object.
 15. The method of claim 14, wherein the model ofthe object is a partial model of the object that comprises the portionof the object, and a distance from the active portable anchor to theaugmented reality device remains five meters or less.
 16. The method ofclaim 15, wherein the partial model of the object is selected from theportion of the object in the field of view of the augmented realitydevice and the portion of the object within a selected distance of theaugmented reality device.
 17. The method of claim 14 further comprising:receiving, by the active portable anchor, a physical anchor position ofthe active portable anchor and the physical object position of theobject as determined by an external tracking system; determining, by theactive portable anchor, the current physical anchor position of theactive portable anchor relative to the physical object position of theobject using the physical anchor position of the active portable anchorand the physical object position of the object; and outputting, by theactive portable anchor, the current physical anchor position of theactive portable anchor relative to the physical object position of theobject to the augmented reality device.
 18. The method of claim 14further comprising: receiving, by the active portable anchor, thecurrent physical anchor position of the active portable anchor relativeto the object.
 19. The method of claim 14, wherein the current physicalanchor position of the active portable anchor relative to the physicalobject position of the object is output to the augmented reality deviceby an output system selected from at least one of a wireless transmitteror a display device.
 20. The method of claim 14, wherein the object isselected from a group comprising a mobile platform, a stationaryplatform, a land-based structure, an aquatic-based structure, aspace-based structure, an aircraft, a commercial aircraft, a rotorcraft,a surface ship, a tank, a personnel carrier, a train, a spacecraft, aspace station, a satellite, a submarine, an automobile, a power plant, abridge, a dam, a house, a manufacturing facility, a building, afuselage, an engine, a wing, a skin panel, a landing gear assembly, anda monument.
 21. A method for orienting an augmented reality devicerelative to an object that comprises a tangible feature, the methodcomprising: identifying, using a tracking system comprising a number ofpositioning locators located off of and away from the object and anactive portable anchor associated with the augmented reality device, acurrent physical anchor position of the active portable anchor; theaugmented reality device using a position of the active portable anchorin a field of view of the augmented reality device for determining anddisplaying augmented reality information from a model of the object in acomputer system on a live view of the object that shows a location ofthe augmented reality information in the model of the object; andoutputting the current physical anchor position of the active portableanchor relative to the object and updating a current model anchorposition relative to the model of the object in an augmented realitysystem.
 22. The method of claim 21, further comprising determining, bythe active portable anchor, the current physical anchor position of theactive portable anchor relative to the object using a physical anchorposition of the active portable anchor and a physical object position ofthe object.
 23. The method of claim 21, further comprising determining,by the active portable anchor, the current model anchor position of theactive portable anchor in the model of the object that correlates to thecurrent physical anchor position of the active portable anchor relativeto the object received from the active portable anchor.
 24. The methodof claim 21, further comprising determining, by an external tracker, thecurrent physical anchor position of the active portable anchor relativeto the object using a physical anchor position of the active portableanchor and a physical object position of the object.